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(54) Input and output communication in a data processing system 

(57) A processor (47) uses a concepts memory area 
(10) to interpret received inputs and to generate coher- 
ent contents-related output communication in text or flu- 
ent speech. lYie processor (47) uses concepts, each be- 
ing a part of program code and a linked basic commu- 
nrcation element. A par^lel recursive communicatkxi 
matrix manager (1 0(a) ) is used in processing Of the con- 
cepts and reference is made to a conceptkxi, learning, 
associative, and utilisation manager (50) to assist in un- 
dcrstandbig inputs, learning and enhancing the knowl- 
edge base, and in developing outputs. The basic com- 
munication elements may trigger associated speech re- 
sources (24) which are retrieved and processed down- 
stream, or rule-based speech outputted via a synthesis- 
er (63). 
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Description 

The invention relates to communication in a data processing system of the •intelligent" type such as an expert 
system. Sucti a system may, for example, be based on the neural network princ^le and may be exclusively mie-based 
s or may include a case based reasoning (CBR) system. 

The purpose o* intelligent systems is to learn or adjust to a multitude of inputs or stimuli and allow a multitude of 
possible results. i.e. there may be a number of "correct' results. These results cause events to occur such as the 
automatic transmission of a signal to a financial institution to cause lodgement of money taan account Development 
ol such systems has progressed to the stage where some can operate in a "hands off" mode for considerable periods 
10 of time. 

However, such features are of little benefit if there is inadequate input and output communication. For output 
communication, in some cases system events can be communicated by display of numbers or numerical furw^tions in 
a clear manner. However, it is increasingly the case that for the important events to be understood, much more com- 
prehensive communication is required. For example, it is desirable that the system be capable of outputting coherent 

IS speech to indicate the events which have occurred, and indeed events which may occur in the future, depending on 
user viputs. It is also desirable that the system be capable understanding inputs with much less human intewention, 
and preferably little or no human intervention. 

Heretofore, the approach to generating output communication has been to select or build pre determined mes- 
sages. For example, in DE4323021 a system accesses, inten'elates, and combines protot^ patterns to genemXe a 

20 composite anage output In JP0731 9538 a building monitoring system is described in which audio data Is selected and 
outputted according to abnormal state information. In EP0052757B a methpd is described in which various tables are 
used by a message build program to display one of a set of pre-detemnined messages. 

In these systems the basic principle is operation of a communication means to select portions of messages and 
to combine Ihem together. This approach is apparently satisfactory for many situations, such as for generating alerts 

£S to nndicate specific and predetermined events, such as a fault in a diskette door. However, where compr^ensive 
system event infonnation is to be communicated tor a complex data processing system it would be necessary to store 
a very large number of message parts arxJ It wouki be unworkable to retrieve arKl process these messages in a mean- 
ingful manner. Further, it would be extremely difficult to foresee all possible communication which may arise in the 
future. The prior systems do not generate coherent communication descrbing any and every possible system event 

30 irrespective of the probability of them arising. 

European Patent Specification Nos. EP0720090A2 and EP0720106A2 both describe systems which provide a 
speech output In these specifications, a "concept" is descn^ied as an entity having successive sub-sets in "ever de- 
creasing circles". The speech output is based on user inputs and the knowledge base is user-deper!denl. While these 
systems are undoubtedly of benefit in some applicatkxis. there is limited capability for providing a coherent output and 

36 for urKJerstarKling a wide range of inputs. 

Further, while the prior art includes many systems for reading inputs such as OCR systenns, little has been done 
to integrate this function with the data processhg. In additkxi. the prtor ait does not generate a coherent and contents- 
related output. 

The invention is directed towards provkJing a data processing system and method to overcome these problems. 
« The inventkxi provides a communk:atkxi method for interpreting communicatron and buikJing output communka- 

tkxi, the method being carried out by a system comprising a processor, a memory, an input device, and an output 
device, characterised m that 

the processor automatcally inteiprets viputs, and 

4S 

the processor generates coherent contents-related communicatkm using a communtoatkxi means linked with data 
processing program code. 

According to another aspect, the invention provides a data processing system comprising a processor, a memory, 
so an input devce, and an output devce. characterised in that- 

the processor comprises means for automatically interpreting inputs, and 

the processor comprises means for generating coherent contents-related communfcation using a communfcation 
ss means linted wiUi data processing system code. 

An important point is that because communicatkxi is generated in-line in a manner linked with the data processing 
program code, the communication Is contents-related and it is therefore possible to provkle coherent communkstion 
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within the conrmes of the knowledge base of the system. 

In one emtxxiiment, the communication means comprises communication elements, each communication element 
being linked to a part of program code. Because the data processing system links the communication elements with 
the associated data processing code as a unit, the communication is bound strongly to the contents ot the communi- 
s cation as it arises from the data processing. 

Preferably, the communk:ation elements are processed in one memory area, either actual or virtual. By processing 
in the one memory area, the processor can efficiently combine the communication elements and also update a knowl- 
edge base. The memory area may be time-deperxlent or time-independent. 

In one embodiment, the prrcessor processes communication elements linked with program code representing 
10 processing results, and communicatkm elements linked with program code used for interim processing before a result 
is determined. This allows very comprehensivo output communkiation as it provides not only the results of data process- 
ing, but also the methodology behind the results. 

Pr^erably, the processor operates according to a matrix manager (or capturing communicatnn elements during 
data processing. This is a very simple tool for capturing communk:ation elements from various difTerent strarrds of data 
IS processing and it in effect synchronises the generation of communication. 

Preferably, the matrix manager directs capture of communication elements tor each of a plurality of data processing 
simulations. There are many situations in which different simulations are required and only one simulation is eventually 
chosen. By c^turing elements from all simulations, there is no delay in generation of the communicatbn after one is 
chosen. 

20 Preferably, the matrix manager is a paralfel recursive function. This is a very effective type of furKton for the task 

involved. -- - 

In one embodiment, the communication elements are triggers which ^e used to retrieve resources including text 
or speech strings. The use of triggers opens up the possibility of a wkJe range of further processing to manipulate the 
output as desired. In one exarrple, the matrbc manager comprise? output communcation rules which are activated by 

25 the triggers to generate processed communicatkm elements. Such mies may comprise prosodic, phonetic, syntactk: 
and semantic processing rules. Such processed elements may be used to retrieve resources to contrd a synthesiser: 
It will therefore be appreciated that by use of triggers, a large degree of addrtkxial processing can be earned out to 
enhartcethe quality of the output in terms erf such things as loudness and intonation, pauses etc. 

In one embodiment, the system comprises speech resources, a speech resources assembly manager, and a 

30 speech resources reference buffer, the assembly manager and the reference buffer performing the steps of :- 

loading bkx:ks of resources to a memory; 

monitoring speech ou^ut and determining when resources of a bkx;k are nearly utilised; 

3S 

toadEng a fresh bkx^k; and 

- clearing the prevkHJS bkx:k when a fresh block is activated. 

40 This is a pariKularty efficient way of handling resources for generation of output communication. 

Preferably, the resources are used to build coherent output communk:atkxi which is further automatically edited 
before output. 

In one embodiment, the processor automatically interprets inputs with reference to a knowledge base whch is 
incremented during processing. Preferably, the knowledge base comprises a plurality of datasets, each accessed by 
4s an associated manager. It has been fourxl that tiy tiaving managers which are responsible for updating particular 
datasets, there is very efficient buikJing of a knowledge base in a nrtanner whereby the knowledge can be easily retrieved 
during processing. 

Preferably, the managers are part of a conception, leaming, associative, and utilisation manager including:- 
so a definilk>n of terms manager which periorms a full text analysis of inputs to define as many terms as possible, and 
a vatklatkMi manager which performs validatkm operations on the defined terms. 

This alkms very comprehensive and flexible interpretatkx) of received communicatkxi to streamline flow of input 
ss communicatkxi. 

In one embodiment, the conceptkxi, leaming, associative, and utilisation manager further comprises:- 

a review specification manager which determines if a term has been used in a similar context before, for validation; 
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and 

a synonym and acronym manager which searches an associated file for synonymous terms. 

5 Preferabty, the processor automatically learns by building the knowledge base, using the knowledge base and 

communication elements as references. 

Inanother embodiment, the conception, leaming, associative, arnJ utilisation manager performs automatic learning 
by adding terms to the datasets as they are recognised to fall within the associated term criteria. Preferably the proc- 
essor prompts user input of data to buitd the knowledge base. 

10 The invention will be more clearly understood from the following description of some embodiments thereof given 

by way of example only with reference to the accompanying drawings in which:- 

Fig. 1 IS an oven/iow schematic representation of an expert system of the invention; 

>5 Fig. 2 is a bkx:k diagram showing the system in more detail; 

Figs. 3 and 4 are still more detailed views showing the system at a tower level; and 

Fig. 5 is a diagram illustrating the manner in which coherent output communicatkx) is generated. 

20 

Referring tn ihe drawings and initially to Fig. 1 there is shown an expert syst^ of the ffiventkxi. As illustrated, the. . 
system comprises a system clock 1 , a working perkxJ manager 2 connected to the clock 1 , data files 3. a resource file 
4, a random access memory (RAM) 5 and an electronic banking manager 6. A bank system 7 communicates with the 
eloctronk: banking manager 6 via a modem 27. The bank s^tem 7 is ntrf part of the expetl system 1 . The system also 

2S includes a processor 47. The system comprises a structure file B an automated cash ftow manager which uses a 
memory area 10. referred to as a concepts memcxy and synthesis area. ARhou^ shown separately, the area 10 is 
part of the RAM 5. The system also comprises disk and CD ROM drives 9. a nrxxirtor 25. a speaker 26, a printer 28 
and a keyboard 29. A synthesiser 63 is connected to the speaker 26. The system is also connected to a networic 62 
from which inputs are received and to whk:h outputs are transmitted. 

30 At the pjrely hardware level, the system is conventional (in this embodiment a microcomputer) and indeed ttie 

method of the invention may be carried out by a wkJe range of hardware systems when suitably programmed. There 
may be multiple ptocessors. 

The stnjcture file 8 is shown in detail in Fig. 2 in combinatkxi with the momoiy vrtiich it utilises. For processing of 
signals or stimuli which arise within the system there is a pattern assembly manager 11 and a pattern conclusion 
3B manager I 20. For processing of stimuli whfch are received from an external source there is a pattem recognition 
manager 31 . a pattem simutatkx) manager 48. and a pattem conclusKin manager II 49. The series of managers 11 
and 20 and the series of managers 31 . 48, and 49 are connected to a oonc^tkm, leaming, and assodatrve utilisation 
(CI-AU) marager 50. 

An important aspect of the memory area 10 ts that the varnus blocks can be indeperKJently activated and the 

40 process fk>w is not necessarily sequenti^ through them. Another important aspect is that the system is capable of 
perfonming data fMocessing and of generating coherent contents-related output communicatran within the confines of 
hs knowledge base. The knowledge base corr^rises various datasets in the data files 3 and associated with the man- 
agers in the CLAU 50. All internal stimuli handled by the managers 11 , 20 and 10a are within this knowledge base. 
However, the managers 31 , 48, and 49 process wrtemal stimuli, some of whKh may not be within the knowledge base. 

45 These managers utilise the CLAU manager 50 to assist with inputs which are not understood, for example, by per- 
forming valklatkxi, synonymfeicronym, or incoherence processing. The "external' managers 31 , 48 and 49 either clarify 
or fail toclarify these signals. If clarified, they are processed as it they were internal signals, for exarrqile by the "intemal' 
managers 11, 20. and 10a. If not clarified, the user is prompted to assist. 

TTius, ttie CLAU manager 50 is generally used by the 'external' managers 31 , 48. and 49 if received inputs cannot 

so be understood and also for broadening the existing knowledge b^e. 

In summary, the input communication is understood by the 'external' managers 31 . 48, and 49. The data processing 
and coherent output communicatkxi may be generated by the 'external' managers 31 , 48. and 49 and/or by the "in- 
temal" managers 11. 20. and 10a. The CLAU manager 50 is used for understanding inputs and for broadening the 
knowledge base. Stimuli whch are generated internally are always understood and are processed by the 'internal' 

55 managers 1 1 , 20, and 1 0a. Therefore, there is also an "understanding" of contents, enabling contents-related speech 
output. 

Before describing how the "internal' and "external' managers operate the folk>wing describes the CLAU manager 
50 in mora detail. 
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51 - Definitkxi at terms manager. This performs a full text analysis of variable electronic input devices in order to 
define as many terms as possible, either by comparing those terms witfi a formerly memorised database or by 
selecting new terms within a predefined range of probability. 

5 52 - Validation manager. This takes the defined terms of the manager 51 and perfomis a validation control. 

53 - Analogy building manager. This determines if an identified term has some meaning for the system's control 
process. For example, if the system is able to identify some terms like Value added lax' (VAT), 'increase' and 
some numeric specificatkin like *18%'. As a consequence, the analogy building manager first controls the current 

10 settings for value added lax, mil recognise that there is some change (in the sense of an irKrease) antl will slate 

that there could be some change from c^er VAT ^lecificatkxis. This will be presented to the user as a question - 
if the irKrease is real and if this has any consequences for the system settings. 

54 - Review specification manager. This determines if an kJentified temi has been used in a different or similar 
'5 context during the past If so, the system analyses under what circumstances this term has been used and wf^t 

decisions were made that time. According to this, the system can decide to proceed in the same way once more 
or not. 

55 - Synonym and acronym manager. This searches within a specific file for s^ionymous terms in orcter to deter- 
20 mine if the selected term has a corresponding item which can be used within the communicatkxi building process. 

This is of importance if the system uses preconfigured speech resources which are addt essed by pointers. If there 
is no predefined speech resource for the cH-iginal item for which the system can identify a synonymous term, the 
pointer is set on the correspomSng speech resource specification. 

56 - Confusion of ideas manager. The confusbn of ideas manager evaluates if the selected terms fiave any cor- 
relation to each other or rf the plausbility is too low, in order to buikJ the communication process out of the terms 
found. 

57 - Associative n;anager. The associative manager compares deferent terms with each other, regarding each 
30 term's surrounding context. This means that a term which the system has found to be strictly usdd in the business 

vocabulary usually will not be mentioned in the communication buikJing process, for example, for medual discus- 
skvi. 

58 - Simulatkxi manager. The smnutation manager is needed, for example for the automatic bank report analysis. 
35 If a corresponding record, for example In the invoces file, is found, the system doesn't stop at this point It continues 

to search for other possibilities, nc^ taking the first choice as the obviously best one. 

59 - Incoherence manager. The incohererK^ manager checks if the words and sentences which are queued up 
by the communicatkxi buiUing process have sufficient correlation to each other, so that the result makes sense 

40 to a human being. 

60 - Plausibility manager. The plausibility manager is acMve in the case of endogenous as well as in the case of 
exogenous incitements, in order to distinguish between two or more possible answers and, if possible, to decide 
for the right choice. On the other side, the plausibility manager also takes control for example in the debit entry 

4s analyser 33, where the total sum is compared with the amounts registered in the past. If there is some suspicious 

de^tion, the system generates a corre^xxiding comment. 

The output from the concepts memory area 10 is fed to downstream modules to complete building of coherent 
contents-related output communication indicating the system events. These include a speech resources assembly 
so manager 21 (c) whch utilises a speech editing process 21 (d) and speech resources 24, and which feeds into a speech 
resources referer>ce buffer 23(b]. The buffer 23^), a grammar rules manager 23(d), a prosode control manager 23 
(e), a semantic control manager 23(f) and a phonetK control manager 23(g) all feed into a speech editing supervisor 
23(c), which in turn controls the speaker 26 either directly or via a synthesiser 83. 

For output of text communication, the ou^ut of the concepts memory area 10 is fed into a text communication 
ss manager 21 (b), which in turn feeds into adocumeni text storage buffer 23(a), in turn driving the monitor 25. In addition, 
there is a management information system manager 61 whteh receives data from the concepts memory area 10. 

Fteferrmg now to Rg. 3, tfiose parts of tfie structure file tor operatkxi in response to internally-generated stimuli or 
endogerraus incitements are illustrated. There is an obligation status manager 13 connected to the pattern assembly 
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manager 11. The pattern assembly manager 11 comprises the following components. These components implement 
data processing and generate coherent contents-related output communication tor the particular type of expert system. 

14 - Recursion manager. The recursion manager cateulates the optimised result, regarding a variable number of 
5 combinable variables. In the example of cash management this means that the manager tries to implement a 

number of obligations within a number of accounts by using the method o( recursive processing. The system 
selects a certain obligation and one ol the defined accounts and controls. H this combination fits. Then it starts 
incrementing the obligations anwunt and tries to combine the drfierent obligations with regard to the given account 
status. When the manager reaches an optimum result, the specified variables are memorised and the search is 
10 performed for the next account. At the end. the system decides lor the highest degree of optimisation and selects 

the most opportune solution. The recursion manager also takes into consideration the available funds and degrees 
the remaining items to the subsequent term ol payment 

15 - Due date manager. The due date manager examines if there are any liabilities, e.g. invoices lor the actual 
IS vwrking period, taking into consideration the defined terms and conditions of payment for each of the invoices. 

16 - Zero-account shift manager. The zero-account shift manager examines the remaining possibilities with regard 
to optimised cash flow - when it is indicated to charge a liability from an account, where the actual amount on the 
credit side is not high enough to pay it from one single account. 

20 

1 7 - Cash discount manager. The cash discount manager analyses which liabilities could be met with regard to 
an offered discount According to the opportune guidelines the system tries to benefit by paying. 

1 8 - Splrttng payment manager. The splitting payment mani=>ger takes into consideratnn that the current amount 
25 cannot be paid from one single account without overdraft and selects two or more accounts, in order to split the 

amount due with regard to an optimised cash ftow. The decision - which amount and which acc<Hjnts are selected 
- depends mainly on the system preferences, whk:h specify if the recipient accepts a split payment or not. 

1 9 - Rate of friterest manager. The rats of interest manager analyses the rates which are due by overdrafts of the 
30 accounts and selects the most favourable combination, which not only depends on the rate itself, but also on the 

actual amount overdraft present on the accounts. In co-operatkxi with the cash discount manager 17 the system 
decides, if it is opportune, to remit payment at present or to defer it to a subsequent term of payment. 

1 2 Bank account allocatkxi nranager. The bank account alkwalion manager selects the most opportune account, 
35 regarding the actual amount on the credK as well as with regard to the interest rate for overdrafts, in order to start 

further caksutatkxis with the first selected liability. 

Fig. 3 also shows those components which use the output of the concepts memory area 10. There is a transactkin 
manager 21 (a) connected to an electronic banking manager 6 which feeds the modem 27 in additkxi to those described 
40 wth reference to Fig. 2. 

Refemng to Fig. 4, particular detail of the strtJCture file in the corKepts memory area Is shown for processing of 
external stimuli or exogenous incitements. These inputs are received tmm a voce recognitkxi system 44, a scanner 
46, the keyboard 29, and an electronc text input system 46. The pattern recognitkxi manager 31 comprises the fol towing 
components. 

45 

32 - Bank account report analyser. The bank account report analyser imports the actual statements of the different 
accounts electronically and starts analysing the contents. It takes not only the documentation into consideration, 
but also the transfened anxxjnt as well as the item's transaction arKJ date specifications Furthermore, tfie bank 
account report analyser starts muttklimensional searches in the database flies, in order to select the most likely 

so records. When there b a number ol more than one possible answer for a cert^n item, the manager carries out a 
plausibility control regarding possible specifications like invok» number, client number, client name, date of invdc- 
ing. date of transactkxi. transferred amount etc. 

33 - Debit entry analyser. The debit entry analyser identifies all transacttons ol debit or credit to an account ac- 
ss cording to registered debit notes. The sy^^ not only examines if there is a registered legrtimisation for booking 

the different debit notes, by also implements different plausibility f unctkxis. It checks if the cun-ent amount exceeds 
the usual range, if the temi of booking is within the estimated timeframe etc. Furthermore it induces the bookkeeping 
process to generate the complete bookkeeping reference. Thereby, the system works according to standard book- 
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keeping rules. 

34 - Money transfer order analyser. "Rie money transfer order analyser works similarly to the debit entry analyser 
33, examining the transfer orders which have been arranged with the different banks. Furthernnre Uie money 

5 transfer order analyser generates the corresponding complete bookkeeping process by allocating the actual 

amount to the predefined booking statements. 

35 - Cash manager payment report analyser. The cash manager payment report analyser evaluates all processes 
that debit the accounts as a consequence of transactbns which were sent by the system to the different banks 

10 according to the cash management calculations one or two days before (conpare endogenous incitements). 

36 - Failure of assignment manager. This manager controls all items which debit or credit an account, but could 
not be allocated by the different managers described above. By using some retrieval procedures, the system 
analyses if similar or analogous processes have taken place in the past and tries toalkx^ate the corresponding item. 

IS 

37 - Date of payment manager. The date of payment manager checks all rwoices according to evaluated bank 
account status and examines if \he date of payment coresponds to the date due. If not, the systems induces a 
reminder process fw the amount due. 

20 38 - Terms of payment manager. The terms of payment manager checks ail invoices according to the evaluated 

bank account status ar>:i examines it the terms of payn^ent i.e. with regard to discounts etc. are met. If not the 
systems induces a reminder process for the remaining amount due. 

39 - Multiple parent manager. The multiple payment manager determines if a payment whKh is credited to an 
zs account related to one or rmre than one invoice. For example, rf a client cdlects several invoices and pays them 

at once, the system has to evaluate the bank account status and has to check whbh invoice records shoukl be 
addressed v.-ithin the database. This is especially difficult when there are discounts which have been taken by the 
client. Of when the client pays several invokes at once, and the amount represents only a part of all unpakJ invoices. 

30 40 - Cash discount manager. Ine cash discount manager checks all invcMces according to the evaluated liank 

account status and examines if the client was entitled to take the discount If not, the system ffiduces a reminder 
process for unpakf amount. 

41 - VcilkJity reference manager. The validity reference manager tries to kJentify if a term whk:h has t>een identified 
35 within a bank status report is valid erKxjgh to be used as a separator term, for exarrfAe in order to distinguish 

between different invoice records. 

42 - Part payment nr^anager. The part payment manager recognises when a liability is partly remitted. The manager 
has to decide if a reminder pnxess shoukJ be initiated, or if the remaining amount due remains within the database 

40 for a further clearing procedure. 

All of the above modules are part erf the pattern recognitkxi manager 31 arKi feed into the pattern simulation 
manager 48. 

Again, the output from the memory area 10 feeds into speech and text output modules as illustrated in Figs. 2 and 
45 3. However, in additnntfiere is an automatic transactkxi manager 43(a), a bankaccount status report manager 43(b), 
and a data reference update manager 43(c). 

As is clear from the above descr^tnn the expert system of the inventkxi is programmed to perform cash manage- 
ment operations in whit^ deciskxis are made as to how money shoukf be managed. In each case there will be a number 
of possible decisnns, many of whk^ are generally correct, artd the system operates to determine the optimum overall 
50 result The resulting event may be kxJgemeni of an entry to a joint account, transmisskm of a signal to the bank system 
for kxigement of money payment of invdces, or directing printing of cheques for payment inv(»ces. 

The inventnn resktes in understarKiing inputs arxJ buiUing output communcation about the events for useis. This 
is achieved in a dynamc and flexible manner which may be in real time. The output communk:atkxi is berth in text and 
in speech. An important feature is that as the data is t>ejng processed, bask: communicatkxi elentents are processed 
ss in combination with linked or associated program code. Together, these may be regarded as a 'concept', whk:h is:- 

(a) the basic program code part relating to the fact matter, result or eviy event for whch there is a need for 
communk^ation, and 
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(b) the corresponding means for communicalicn. 

Thus, il will be rwted from Fig. 4 that the managers of the pattern recognition manager 31 perform data processing 
with integrated communication operaltcns using concepts. 

5 As stated above, the system generates coherent output communication within the confines of its knowledge base 

- the data files 3 and the managers 51 -60 in the CLAU manager 50. This knowledge base is used to determine if inputs 
can he processed, and some inputs are used to burW the knowledge base. The learning functton is automatic and it 
uses the <x)ncepts arxl the knowledge base as references. It also uses interactive user inputs. A simple example of 
learning is addition of received words to a synonym file by the synonym and acronym manager 55. The word is added 

10 to the file after confirmatkwi by a user. On the other hand automated learning may take place by addition of a word to 
a synonym table if the relationship is clear after operation of the review specificatkm manager 54. Another example of 
automated teaming is addition of a word which is known from an algorithm to a data file. 

The generation o* coherent output communication is illustrated in Fig. 5. The arrow A indicates the program flow 
of the processor using lines 70 of code. Some of the lines 70 have associated canmunication elements, in this em- 

is bodtment resource triggers 71 . By virtue of the fact that the triggers are linked inherently with the program flown they 
ccmbme together in sequence to achieve correct syntax and coherency. In ottier words, the output convnunication is 
linked directly with the program flow and is thus always coherent and has correct syntax within the confines of the 
knowledge base of the program fkwt The communication matrix rranager lOe captures the flow of triggers and es- 
sentially act as a transfer buffer between the data processing managers and the communksition control manager 21 

20 Because the communicatton is directly linked with the program flow in this manner, it is contents-related, compre- 

hensive, and flexible. For example, the same words are used to provide different meanings in different contexts. The 
communication buikJing is setf-organismg with <^ration of ttie matrix manager and a quantitatively and qualitatively 
unlimited variety b achieved. Thus, the output communication is not foreseeable with regard to qusnlrty or contents. 
There is also Hexibtiity with re^rd to initiatkxi of a communcation - it may be the user or ttie system. 

25 In more detail, the basic elements <rf communicatton may be single letters, combinations of letters, words, combi- 

nations of words, sentences, contfsinatkms of sentences, phonetic expressions, numbers, combinatbns of numbers, 
figures, or combinations of figures. Generation of concepts in this manner allows communication in both directkxis. 
The system coherently generates conrvnunicatkvi to output onformation abcxjt the events, and il coherently evaluates 
received data and communicates the results to the user. Any suitable programming language nray be used tor the 

30 program code. The important point is that the generic elements for communicatron, or alternatively codes which trigger 
corresponding communication elements, are associated with the corresponding program code to fonn concepts. There- 
fore, a varoble as well as dynamic connectivity between syntactic elements is provided for. This may be acconplished 
in one embodiment by specification of entry and exit points to enable subclauses, inteijectkms and even specificaticxt 
of grammar mles which are essential. 

36 In a further embodiment mutttdimenstonal comnrands with descriptive elements for intonation, cyntax, phonetics, 
prosodk: control and semantics as well as intormation elements describing the contents are linked with the communi- 
cation matrix manager, thus providing complete DifonraAion needed for ccHitents-related speech output 

A system bound acteiptive piosod'ic method is integrated whereby the system itself recognises, for example, unusual 
events and as a consequence of this alters the pronunciatkxi to sb^ess the unusual situation. 

40 It is preferable that the communicatton elements are situated within the same memory area In the ^>eclftc example 

of speech, a preferred embodiment is to store the speech resources elsewhere and trigger them witti the elements for 
communication within the concepts niemory area, thus altowing speech editing with regard to semantics, prosodic and 
gramnrar control. 

This basic architechjre also alkiws the resources to be stored remote from the concepts memory area 10. either 
45 in a \oca\ or remote network. Thus, there is little networtc traffic in relatkm to the volume of output communcation as 
only ti'iggers (and/or a single matrix) are transmitted over the network. 

In more detail, the speech resources are triggered in a manner whereby they are not kiaded in sequence, but 
instead as bkx:ks of twenty resources at one time. This cables fluent speech and avoids meoKtry overflow. This is 
achieved by operation of an index table of the speech resources 24 monitoring the process and recognising when a 
50 btock of resources is nearing its end. It toads the next btock and after the new bbck has started it clears the memory 
of ttie previous block. Sentences are formulated with an open end. Thus, they may be of any vanab\e length. A time- 
dependent handling of the speech in btocks is nxxe piactnal than byte-related handling as required by the prior art 
such as EP 0052757B. As the matrix contains all contents-related information needed for a human-like speech the 
matrices are used in some embodiments together with the communication control manager 21 to command a synthe- 
ss siser 63. In these emtxxJiments the speech output is rule-based and not data-based as described for some other 
embodiments refen-ed to above. 

The system handles data differently depending on whether or not it originated within tiie system (endogenous) or 
externally of the system (exogenous). These differences are illustrated h Figs. 2, 3 and 4. The important point is that 
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ttie endogenous data does not need to be recognised before it is processed. For the exogenous stimuli the pattern 
recognition manager 31 performs the necessary reco^itton functions before pattern simulation and conclusion. 

Within the concepts memory 10. the CLAU manager 50 con^ects. evaluates, or changes communicalion buitt up 
or recognised by the mar^gers in the memory area 10 (see Fig. 2). For example, the manager 55 will not reject an 

S exogenous incitement which, with the exception of one word, is othenvise coherent with regard to its contents. In other 
words, it is understood when and if the word in question is a synonym to another which is stored in a concept in the 
menwry are 1 0. A more comprehensive description of the functions of the nanager 51 to 60 which make up the CI-AU 
manager 50 is given above. The important point is that they interact with the expert system managers 1 1 , 20, 31 , 48 
and 49 to assist in communication understanding and generation. 

10 As stated above, received information is mterpreted, and output comrr.unication is generated in an integrated 

manner during data processing, using concepts. Communicalion is understood and built using basic communication 
elerrrants which should not be confused with pre-defined messages or alerts, because they are used in understanding 
and building coherent communication in an entirety flexible manner "Rits communication is purpose-built from the 
elements of communication linked to the basic program code during data processing with the assistance of the matrix 

fs manager 10a. 

The first part of the Appendix (A1 ) provides code samples of the pattern assembly manager 11 , which recognises 
ir^uts. This allows, for excunple, automatic understarxiing of who has paid monies, for which reason, and when. 

Tho first page of the Appendix includes (feta processing code relating to account balances and split payments. 
Concept syntlwsis is then started by the first line of the second page, narT.e)y "A. The line "B activates a comment 

^ buitdhg procedure, and the line *C activates a speech building procedure. The data is indicated by a trigger "ArrlnsE- 
temStr (»0ArrSprache;''522'') which refers to Array Instruction Element string 522 in the speech resources 24. The 
Apperxlix includes many eiqplanatory comments expteuning to the reader the communtcation building process. It will 
be apparent tfiat both commeni (text) and speech communications are generated simultaneously. The lines following 
the explanatory comments after line *C include triggers to additional resources, and these are in turn followed by more 

2S progrsun code. The subsequent lines T), and ''F start procedures as set out and are folbwed by more triggers. It 
will be noted tfiat the triggers and program code are blended togetfier in a very flexible manner to allow simultaneous 
data processing and generation erf convnunication. The Appendix A2 gives ein example of code for the pattern recog- 
nition nranager 31, which deals vnth external stimuli ^an, the same princqile applies. In this example, the line *G 
activates a concept synthesis procedure and a text comment building procedure is ac^vated by tine A speech 

30 building process is activated by line 1 and this is followed by triggeis for generation of output communication. The 
functions of the above three is repeated for the lines *J, *K, and *L with dif erent data. 

The matrix manager 10a is an entity which contributes to the building p^^ocess of communccatton in an onlinemode. 
It is a p£ualtel recursive functicm whic^i is constantly available to the managers with which it operates. Its function is 
not dependent on the presence of a referer>ce t^le. This is due to the fact that the elements are by no means limited, 

35 as with the prior art. For example, it interacts with the Bank Account Report Analyser 32 by selecting several elements 
within the analyser 32 arKl comparing them to stored data. The plausibility is then chedted and if fou nd true all elements 
analysed will be stored for later output. Accordingly, new elements are stored in a manner which enables the system 
to team by wkiening its knowledge base. The rrtatrix manager 10a may therefore be regarded as a general tool used 
in the corx^pts memory area to assist in prxx^ssing elements or corv^ts for sorting, corr^iarison. ariput or output. 

40 This is a^ieved by constantly scrolling through the elements or concepts for communication buiUing. At Itssffnplest, 
the matrix manager 10(a) captures communication elemente arxJ transfers them to the communication control manager 
21 in a certain order. If the data processing inwlves multiple simulations, elements for each simulation are stored and 
are immediately available for use. A more complex system may include rules in the matrix manager which receive the 
elements as inputs and generate processed elements for transfer on. For example, there may be prosodc, phonetic, 

45 syntactic, and semarrtic specifnation mies and also speech pause rules. A single trigger may be used by a nurr^er of 
rules to generate processed elements whk^ combine together to give a comprehensive output 

A synchronous formation al the matrix enables parallel in-line and immediate output of text and/or speech. On the 
other hand, asynchronous formation is preferred if there is no need for a time-dependent immediate cnitput. 

It will be appreciated that the invention provides a method and system for understanding inputs and generating 

so cc^ierent communication outputs in an entirety flerable manner. Each communication output is entirely new as it is buitt 
up from small elements and corresponds coherently to the events which occurred. Because the corrOTiunicatkxi is buitt 
up during processing in an integrated manner, it is contents-related and it informs the user ndl only of occurrence of 
an event, but also why the event occurred and how the event affects other events. For example, if the event is the fact 
that no processing result is possible, then the output communication will specify why there was no result. This is ex- 

ss tremety important for users of an expert system. 

It will also be appreciated that there is considerable flexibility in communkstion output For example, there are no 
pre-detined endings to sentences - text is added until a coherent communication reflecting the events is achieved. The 
communicatbn is thus open-ended and this complex inforrratton may be communicated at "any length". The only 
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t^les which are used are ones storing basic numerals 0 to 9 and the alphabet - communication being dynamically 
understood and built online. In effect, what the Bivention has achieved is to combine Input and output communication 
with the core processing in order to simulate the ability of a human to understand what processing is taking place and 
to communicate on the basis of this understanding. This is in contrast to the prior art, in which elements of output 
5 cxxnmunication are formulated according to a relatively rigid structure. 

In summary, the technical communication steps achieve the result of a fluent and coherent output - just like a 
person explaining the events. For example, a graphical representation of a talking man may be generated on screen 
while the coherent speech is outputted. The comment text may be simultaneously displayed. 

10 
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APPENDIX 

A ppendix A1 

Concepts Synthesis within the Pattern Assembly Manager for accounts, endogenous 
incitemenis 

c_boolean(Ov Allocated) 

OvAlIocated:=False 

SEARCH for accounts 

where the balance is greater than 0 before payment of one of the invoices 
where the balance is greater than 0 after the payment of one of the invoices 
where the balance is greater than the predefined minimum after payment 
where the cash flow automatisation status is set to true 
If the number of the according accounts is greater tfian 0 
0^yCond:="Paying within the positive range** 
OvAllocated:=True 

End if 

If ((OvAllocated=False) & (OvSpliCancel=False)) 
SEARCH for accounts 
vdiere the balance is greater than 0 before tiie payment of one of the 
invoices 

^lere the cash flow automatisation status is set to true 
If (the sum of amounts of the selected accouDts>=OvAmountAkt) 
FK_Prior_Num TPlus") 
If(OvAblNight=False) 
If ((OvFKNumbeK=OvSplNumMax) & (OvSpliBool=True)) 
OvPayCond~"Splitting" 
OvAllocated:=True 
Else 

OvPayCond:FK_PriorityPl ($1) 
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*A SUrt Concept Synthesis 

Start Splitting Payment Manager 

Check, if the primary settings for splitting payment are followed 
*B Start the comment bmlding process and add thefottowings terms: 

• the invoice number of the current invoice 

• the offered cash discount 

• Offer the user to switch the cunent settings according to the needed 

status 

*C Start the speech building process and add the follow, ref. numbers: 
ArrlnsEleraStr (»0ArrSprache;"522") 
Speech resource 522 standing for tiie character "a" 

CharAnalyscl ( »OArrSpfachie;[BankStatusReport]ID) 

procedure CharAnalyscl airalyses the parameter, vrhich is presented by the 
variable [BankStatusReportjID. According to tiie length of the term and 
the digits within the term the procedure CharAnalyscl a{^nds the 
pointers for the diffeicnt speech resources (for example Sp-R. 501 for digit 
**r, Sp.R- 552 for character "a" and so on). Every pointer to a certain 
speech resource is added to Ae speech-array OaSprache. 

ArrlnsEleraStr (»0AiTSprache;"641") 

ArrlnsElemStr (»0AirSpnichc;"704") 
CbarAnalyse3 (»OArrSpTache;$Amount) 

procedure CharAnalysc3 analyses a real number, which is presented by 
Ae variiible *^Amount''. From digit to digit the procedure assigns a 
speech resource and adds the pointer to this resource to the speech-array 
CuSprache. 
ArrhisEIemStr (»OArrSiKache;"l 12") 

ArrlnsElemStr (»OArTSprache;"103") 

ArrlnsElemStr (»OArrSprache;**10n 

End if 

Else 

If ((OvFKNumber<=OvSplNumMax) & (OvSp1iBool=True)) 
OvPayCond:- ^Splitting Payment" 
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Ov Allocated -True 
End if 

End if ^ 
End if 
End if 

If (OvAlIocated=False) 
Search for all accounts 

where the balance is greater than 0 before the payment of one of the invoices 
where the cash flow automatisation status is set to true 
If the sum of amounts of the selected accounts>=OvAmountAkt) 
FK_Prior_Num ("PlusMinus") 
If(OvAblNi^t=False) 
Case of 
:(OFKNumber>l) 
If(OvSpliCancel=False) 

If ((OFKNumber<H>vSplNumMax) & (OvSpliBool=TTue)) 

OvPayCond:="Splitting Payment" 
0\'Allocated~Tnie 
Else 

OvPayCond.-=FK_PriorityP2 ($1) 
*D Start Concept Synthesis 

Start Splitting Payment Manager 
*E Start the comment buiUUng process and add the following terms: 

• the invoice number of the current invoice 

• the offered cash discount 

• the difference between the offered cash discount and the needed 

payment according to ^ rate of interest, if the invoice shoult be 
payed now 
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Start the speech building process and add the follow, ref. numbers: 

ArrlnsElemStr (»0AiTSprache;"604") 
Speech resource 552 standing for the partial work "Split" 

CharAnalysel (»OSprache;[BankStatusReport]ID) 
procedure CharAnalysel analyses the parameter, which is presented 
by the variable [BankStatusReport]ID). According to the length of the 
term and the digits within the term the procedure CharAnalysel 
appends the pointer for the different speech resources (for example 
Sp-R. 501 for digit T', Sp.R. 552 for character "a" and so on). Every 
pointer to a certain speech resource is added to the speech-array 
OaS[vache. 

ArrlnsElemStr (»OATrSprache;"70r ') 

ArrlnsElemStr (»0ArTSprache;'*U5") 

CharAnalyse3 (»OArrSprache;$DiffeTence) 

procedure CharAnalyseS analyses a real number, which is presented 
by the variable "SDiflference". From digit to digit the procedure 
assigns a speech resource and adds the pointer to this resource to the 
speech-array OaSprache. 
ArrlnsElemStr {»0ArrSprache;"12r*) 

ArrlnsElemStr (»OArTSprache;*'89yO 

ArrlnsElemStr (»0ArrSprache;"222") 

End if 

End if 

(OvFKNumber=l) 
OvPayCond:=*^plitting Payment" 
OvAllocatedTnie 
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Appendix A2 

Concepts Synthesis within the Pattern Recognition Manager for 
ezogeoeous incitements 

C_TEXT(OvComment) 

C_LONGINT($i) 

OvCommentr^"" 

USE SET("BankStatusReportSet") 

FIRST RECORD([BankStatusReport]) 

RecLoadWait (»[BankStatusReport)) 

For ($i; 1 ;Records in seiection([BankStatusReport])) 

[BankStatusReport]DocuText:=SpaceDeIete ([BankStatusRepon]DocuText) 

Case of 

: ([BankStatusReport]Amount>0) 
If (AuszugFilterRA ("NichtBeNumt->=Tnie) 

[BankStatu5Report]Detected.^Tnie 

SAVE RECORD([BaiikStatusRcport]) 
Else 

If ( AuszugFilterZE =Tnie) 

[BankStatiisReport]Detected:=TTue 

SAVE RECORD([BankStatusReport]) 
Else 

If (AuszugFilterRA ("BeNumt")=True) 

[BankStatusRepc^]Detected~Tnie 

SAVE RECORD([BankStatusReportl) 
Else 

*G Start Concept Synthesis 

*H Start the comment building process and add the following terms: 
lf(Length(OvComment)#0) 

OvCommenCOvComment+OCR+OCR 
End if 
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OvCommem:=OvComment+"Filtervorgang: Bankauszug Nummer" 

+[BajikStatiisReport)II>*-":"+OCR 
OvComment;= OvComment+"Eine Zourdnimg ist bislang noch nicth 

maglich!" 

*I Start the speech building process and add the follow, ref. numbers: 

AiTlnsElemStr (»OAiTSprache;"552") 
CharAnalysel (»OArTSfMache;[BankStatusReport]ID) 
ArrlnsEIemStr (wOAirSprachcf 553") 
End if 
End if 
End if 

: ([BankStatusReportJAmountO) 
If (StatusFiltDebit -True) 

|BankStatusRfiport]Detected:=Tnie 

SAVE RECORD([BankStatusReport]) 
Else 

If (StatusFilterZA =True) 

[BankStatusReport}Detected:=True 

SAVE RECORD([BankStatiisReport]) 
Else 

*J Start Concept Synthesis 

*K Start the comment huUMttg process and add the following terms: 
lf(Length(OvCoinment)#0) 

OvConiment=OvComment+OCR+OCR 
End if 

OvComment:=OvComment+"Filtervorgang: Bankauszug Nummer" 
+[BankStatusReport]ILH":"+OCR 

OvComment:=OvComment+"Eine Zuordnung ist bislang nicht mdglich!'* 
*L Start the speech buUding process and add the follow, ref numbers: 
ArrlnsEIemStr (»OArrSiwachc;*'552") 
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CharAnalysel (»OArrSpTache;[BaiikStatusReport]ID) 
ArrlnsEIemStr (»0ArrSprache;"553") 
End if 
End if 
End case 

NEXT RECORlXrBankStatusReport]) 
RecLoadWait (»[BankStatusRqx>rtJ) 
End for 



Clalmfi 

20 

I. A cqpfvnunication method for interpreting communication and buitding output communication, the method being 
carried out by a system comprising a processor (47), a memory [3. 5, 10), an input device (9. 29, 44, 45, 46, 62), 
and an output device (25, 26, 28, 63), characterised in that: 

zs the processor (47) automatically interpretis inputs, and 

the processor generates coherent contents-related communication using a communication means linked with 
data processing program code. 

30 2. A method as claimed in claim 1, characterised in that the communication means comprises communication ele- 
ments r/1 ), each communication element being linked to a part of program code (70). 

3. A method as clamed in claim 2, characterised in that the communicatkxi elements (71) are processed in one 
memory area (10), either actual or virtual. 

4. A method as claimed in claims 2 or 3, characterised in that the processor (47) processes communtcation elements 
(71) linked with program cods representing processing results, and communeation elements (71) linted with pro- 
gram code used for interim processing before a result is determined. 

40 5. A method as claimed in any preceding claim, characterised in that the processw (47) operates according to a 
matrix nnanager (10(a)) for capturing communication elements (71 ) during data processing. 

6. A method as claimed in claim 5, characterised in that the matrix manager (1 0(a)) directs capture of communKatkm 
elements (71) for each of a plurality of data processing simulations. 

4S 

7. A method as claimed in claims 5 or 6, characterised in that the matrix manager (10(a)) is a parallel recursive 
function. 

8. A method as claimed in any of claims 2 to 7, characterised in that the communication elements (71) are triggers 
so whnh are used to retrieve resources (24) including text or speech strings. 

9. A method as claimed tn claim B, characterised in that the matrix manager (1 0(a)) comprises output communcatkm 
rules which are activated by the triggers to generate processed communicatkxi elements. 

55 10. A method as claimed in claim 9, characterised in that said rules comprise prosodic, phonetic, syntactic and semEintic 
processing rules. 

II . A method as claimed in claims 9 or 10, diaracterised in that the processed elements are used to retrieve resources 
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to control a synttiesiser (63). 

12. A method as claimed in any of claims B to 11 , characterised in thai the system comprises speech resources (24), 
a speech resources assembly manager (21(c)). and a speech resources reference buffer (23(b)), the assembly 
5 manager and the reference buffer performing the steps of :- 

loading blocks of resources to a memoy; 

rrwnitoring speech output and determining when resources of a block are nearly utilised; 

10 

toadtng a fresh btocK and 

clearing the prevkujs block v/Uen a fresh block is activated. 

IS 1 3. A method as claimed in any of claims 8 to 1 2, characterised in that the resources are used to buiki coherent output 
communication which is further automatically edited (23(d) - 23(g)) before output. 

14. A method as claimed in any preceding claim, characterised in that the proces sor automatically interprets inputs 
with reference to a knowledge base (3, 51-60) whch is incremented during processing. 

20 

15. A method as claimed in claim 14, characterised in that the knowledge base comprises a plurality of dalasets (3, 
51-60), each accessed by an associated manager. 

16. A method as claimed in claim 1 5. characterised in that the managers are part of a conception, leaming, associative, 
2S and utilisatkm manager (50) including:- 

a definition of terms nraanager (51) which performs a lull text analysis of inputs to define as many terms as 
possible, and 

30 a validation manager (52) which performs validatkxi operatkxis on the defined terms. 

17. A method as claimed m claim 1 6, characterised in that the conceptkxi, teaming, associative, and utilisation manager 
(50) further comprises:- 

35 a review specrftaatkm manager (54) whch detemiines if a term has been used in a similar context before, for 

validatkxi; and 

a synonym and acronym manager (55) whkih searches an associated file for synonymous terms. 

40 18. A method as claimed in any of claims 1 4 to 1 7. characterised in that the processor automatically learns by buikjing 
the knowledge base, using the knowledge base and communcatkm elements as references. 

19. A method as claimed in any of claims 16 to 18. characterised in that the conceptton, leaming, associative, and 
utilisation manager (50) performs automatic leaming by adding terms to the datasets as they are associated with 

45 term criteria. 

20. A method as claimed in daim 1 9, characterised in that the processor prompts user input of data to build the knowl- 
edge base. 

so 21. A data processing system comprising a processor (47), a memory (3, 10, 5), an input device (9, 29, 44, 45, 46, 
62), and an output devue (25, 26, 27, 63), characterised in that- 

the processor (47) comprises means for automatically interpreting inputs, and 

6S the processor comprises means for generating coherent contents related communicatkxi using a communi- 

cation means linked with data processing system code. 

22. A data processing system as claimed in claim 21, characterised in that the communKatkxi means comprises 
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communication elements (71), each communication element being linked to a part of program code. 
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